Skip to content

feat(runtime-interface-client): Add Lambda-Runtime-Invocation-Id support for cross-wiring protection#624

Open
vip-amzn wants to merge 1 commit into
mainfrom
feat/invocation-id-cross-wiring
Open

feat(runtime-interface-client): Add Lambda-Runtime-Invocation-Id support for cross-wiring protection#624
vip-amzn wants to merge 1 commit into
mainfrom
feat/invocation-id-cross-wiring

Conversation

@vip-amzn

Copy link
Copy Markdown

Summary

Add Lambda-Runtime-Invocation-Id header support for cross-wiring protection.

The RIC now echoes the invocation ID received from RAPID on /next back on /response and /error, enabling RAPID to detect and reject stale responses from timed-out invocations.

Problem

On Lambda Managed Instances (LMI) and On-Demand (OD), when an invoke times out, the runtime process continues running in the background. If a new invoke arrives with the same requestId, RAPID accepts it. The still-running old invocation eventually posts its response, and RAPID matches it to the new invoke — delivering the wrong response (cross-wiring).

Solution

RAPID sends a unique per-invoke identifier via Lambda-Runtime-Invocation-Id header on /next. The runtime echoes it back on /response and /error. RAPID validates the match before accepting the response.

Changes

  • aws-lambda-cpp-0.2.7/runtime.h: Added invocation_id field to invocation_request, added param to post_success/post_failure/do_post
  • aws-lambda-cpp-0.2.7/runtime.cpp: Parse header in get_next(), add to curl request in do_post() when non-empty
  • NativeClient.cpp/.h: JNI reads invocation_id from C++ response, passes to Java; postInvocationResponse accepts nullable invocationId
  • InvocationRequest.java: New invocationId field
  • NativeClient.java: Updated native method signature
  • LambdaRuntimeApiClient.java/Impl: reportInvocationSuccess/reportInvocationError accept invocationId
  • AWSLambda.java: Thread invocationId through runtime loop
  • Unit tests: Updated assertions for new parameter
  • Version bump: 2.11.0 → 2.12.0

Backward Compatibility

Fully backward compatible in both directions:

  • If RAPID doesn't send the header → RIC doesn't see it → doesn't echo → no behavior change
  • If RIC doesn't echo it (old version) → RAPID skips validation → no behavior change

@rudraroop rudraroop left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall - my major point of feedback would be adding a test for the positive path where invocation IDs are non-null

Have added some casing/hardcoding related comments - whatever I could see

headers.put(ERROR_TYPE_HEADER, error.errorType.getRapidError());

if (invocationId != null) {
headers.put("Lambda-Runtime-Invocation-Id", invocationId);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the other headers in this file are declared as constants at the start of the file. I would suggest keeping that uniform and making a constant above for this one as well

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1


LambdaError lambdaError = new LambdaError(errorRequest, rapidErrorType);
lambdaRuntimeApiClientImpl.reportInvocationError(requestId, lambdaError);
lambdaRuntimeApiClientImpl.reportInvocationError(requestId, lambdaError, null);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have one test for reportInvocationError which tests a positive path i.e. one where the Lambda-Runtime-Invocation-Id is actually getting passed instead of null

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added reportInvocationErrorWithInvocationIdTest in LambdaRuntimeApiClientImplTest.java.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same feedback for positive path test inclusion as in this file LambdaRuntimeApiClientImplTest.java‎

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@vip-amzn vip-amzn Jul 23, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added testInvocationIdIsPassedToReportError and testInvocationIdIsPassedToReportSuccess tests.

headers = curl_slist_append(headers, "transfer-encoding:");
headers = curl_slist_append(headers, m_user_agent_header.c_str());
if (!invocation_id.empty()) {
headers = curl_slist_append(headers, ("lambda-runtime-invocation-id: " + invocation_id).c_str());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should use the header constant here instead of hardcoding again?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

headers.put(ERROR_TYPE_HEADER, error.errorType.getRapidError());

if (invocationId != null) {
headers.put("Lambda-Runtime-Invocation-Id", invocationId);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@@ -41,6 +41,7 @@ static constexpr auto COGNITO_IDENTITY_HEADER = "lambda-runtime-cognito-identity
static constexpr auto DEADLINE_MS_HEADER = "lambda-runtime-deadline-ms";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

general question we need to investigate. Why does the jni module source is in a version tagged folder (aws-lambda-cpp-0.2.7), do we expect this to change in the current release model? Is this intentional?

This can also be interesting for @fabisev

headers = curl_slist_append(headers, "transfer-encoding:");
headers = curl_slist_append(headers, m_user_agent_header.c_str());
if (!invocation_id.empty()) {
headers = curl_slist_append(headers, ("lambda-runtime-invocation-id: " + invocation_id).c_str());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1


LambdaError lambdaError = new LambdaError(errorRequest, rapidErrorType);
lambdaRuntimeApiClientImpl.reportInvocationError(requestId, lambdaError);
lambdaRuntimeApiClientImpl.reportInvocationError(requestId, lambdaError, null);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 65.38%. Comparing base (69b9b2b) to head (371508e).

Files with missing lines Patch % Lines
.../client/runtimeapi/LambdaRuntimeApiClientImpl.java 85.71% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##               main     #624   +/-   ##
=========================================
  Coverage     65.38%   65.38%           
- Complexity      211      212    +1     
=========================================
  Files            34       34           
  Lines           988      991    +3     
  Branches        142      143    +1     
=========================================
+ Hits            646      648    +2     
  Misses          290      290           
- Partials         52       53    +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vip-amzn
vip-amzn force-pushed the feat/invocation-id-cross-wiring branch from abe563d to 69cdf71 Compare July 24, 2026 08:31

@rudraroop rudraroop left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks for the changes

@vip-amzn
vip-amzn force-pushed the feat/invocation-id-cross-wiring branch 2 times, most recently from 93fa584 to 25fbaa6 Compare July 24, 2026 14:17
FROM public.ecr.aws/amazoncorretto/amazoncorretto:8

# Install docker and buildx extension
RUN amazon-linux-extras enable docker && \

@vip-amzn vip-amzn Jul 24, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failing smoke test on Java RIC PR:

What happened:

The Dockerfile.agent at test/integration/codebuild-local/Dockerfile.agent (on main: https://github.com/aws/aws-lambda-java-libs/blob/main/aws-lambda-java-runtime-interface-client/test/integration/codebuild-local/Dockerfile.agent#L4) uses amazon-linux-extras enable docker which doesn't exist on AL2023.

Proof — same Dockerfile, different base OS:

Successful job (before Jul 21): sha256:cbe7fe... → Amazon Linux 2
Failed job (after Jul 21): sha256:f4ee79... → Amazon Linux 2023

Corretto announcement:

https://github.com/corretto/corretto-docker#notice-default-images-moved-from-amazon-linux-2-to-amazon-linux-2023

Amazon Linux 2 (AL2) is now end of life. The default amazoncorretto images (for example amazoncorretto:8, amazoncorretto:11, amazoncorretto:17, amazoncorretto:21 and latest) are now based on Amazon Linux 2023 (AL2023). Corretto 22+ images were already AL2023 based.

Who uses it:

Only make test-smoke / make test-integ (local CodeBuild agent for integration tests). The actual make test and make build targets that compile and run unit tests pass fine — those don't use this Dockerfile.

Fix options:

  1. Change base to amazoncorretto:8-al2 (fallback tag, temporary)
  2. Remove amazon-linux-extras enable docker line (docker is in default repos on AL2023)

Going ahead with Option 2.

On AL2023, xargs is no longer included by default (it was part of findutils on AL2 which was pre-installed). The test script uses xargs but the codebuild agent image doesn't have it.

…ort for cross-wiring protection

Echo the invocation ID received from RAPID on /next back on /response
and /error, enabling RAPID to detect and reject stale responses from
timed-out invocations. Fully backward compatible.
@vip-amzn
vip-amzn force-pushed the feat/invocation-id-cross-wiring branch from 25fbaa6 to 371508e Compare July 24, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants